The present application generally relates to a parallel computing system. More particularly, the present application relates to synchronizing processors in the parallel computing system.
A parallel computing system includes a plurality of processors that can concurrently operate. These processors may cooperate together to perform a certain operation, e.g., an arithmetic operation. To work together, processors in the parallel computing system can be synchronized according to a global clock signal.
Modern processing systems have clock frequencies in a multi-GHz range. This results in communications paths between processors necessarily involving multiple clock cycles. Additionally, the clock frequencies in modern multiprocessor systems are not all exactly equal, as they are typically derived from multiple local oscillators that are each directly used by only a small fraction of the processors in the multiprocessor systems. Having all processors utilize the same clock may require that all modules in the system receive a single global clock signal, thereby requiring a global clock network. Both the lack of a global clock signal and the complexities of synchronization of chips when communication distances between chips are many cycles may result in an inability of modern systems to exactly synchronize.